Some Issues in the Testing of Computer Simulation Models

The testing of simulation models has much in common with testing processes in other types of application involving software development. However, there are also important differences associated with the fact that simulation model testing involves two distinct aspects, which are known as verification and validation. Model validation is concerned with investigation of modelling errors and model limitations while verification involves checking that the simulation program is an accurate representation of the mathematical and logical structure of the underlying model. Success in model validation depends upon the availability of detailed information about all aspects of the system being modelled. It also may depend on the availability of high quality data from the system which can be used to compare its behaviour with that of the corresponding simulation model. Transparency, high standards of documentation and good management of simulation models and data sets are basic requirements in simulation model testing. Unlike most other areas of software testing, model validation often has subjective elements, with potentially important contributions from face- validation procedures in which experts give a subjective assessment of the fidelity of the model. Verification and validation processes are not simply applied once but must be used repeatedly throughout the model development process, with regressive testing principles being applied. Decisions about when a model is acceptable for the intended application inevitably involve some form of risk assessment. A case study concerned with the development and application of a simulation model of a hydro-turbine and electrical generator system is used to illustrate some of the issues arising in a typical control engineering application. Results from the case study suggest that it is important to bring together objective aspects of simulation model testing and the more subjective face- validation aspects in a coherent fashion. Suggestions are also made about the need for changes in approach in the teaching of simulation techniques to engineering students to give more emphasis to issues of model quality, testing and validation

Powering Future Transport in Scotland: A Review for the Scottish Association for Public Transport

This report discusses energy costs and emissions associated with transport in Scotland and reviews options for future power sources for different modes of public transport. Transport provides a major contribution to greenhouse gas and other harmful emissions worldwide and efforts to reduce these are important for all forms of public transport, as well as for private cars and for the movement of freight. The effects of transport policy decisions are recognised, increasingly, as being very important for the electricity supply industry at national and local levels, largely because of the growth in the numbers of electric and hybrid road vehicles. Moving from oil to low carbon energy for transport raises important issues for electrical power generation and distribution systems in addition to challenges already being faced by the electrical power industry as the proportion of generating capacity involving renewables increases. The report starts by considering current energy costs and emissions for different forms of passenger transport and then outlines some current developments in areas such as internal combustion engine technology, battery storage systems and hydrogen fuel cells. Systems involving short-term energy storage and recovery of energy that would otherwise be dissipated as heat during braking are also discussed. Such systems generally involve the use of super-capacitors, flywheels or hydraulic devices. References are provided to the sources of data used in the analysis carried out for this review and, also, to sources of information about relevant developments in science and engineering. For all the new developments mentioned, there is a brief review of some transport applications in the United Kingdom and elsewhere. The possible impact of autonomous vehicles on future car ownership is still not known and the effects of this technology on public transport remain uncertain. As well as discussing autonomous road vehicles, the report makes brief mention of the potential of autonomous systems and increased automation for rail transport and for tramway operations. The benefits of further conventional railway electrification are reviewed in terms of energy usage, costs and emissions and the advantages of a more integrated approach to the provision of public transport in Scotland are emphasised. The value of using mathematical modelling and simulation methods to explore options in transport systems developments and planning is discussed, and the importance of testing simulation models in ways that are appropriate for the intended application is emphasised. This review presents the first results from a continuing study which was started in 2018 and is intended to provide information that should be relevant for those involved in decision-making in Scotland at the time of publication. The quantitative information contained within it clearly needs to be updated on a regular basis. The review concludes with recommendations for the Scottish Association for Public Transport about possible priorities for its efforts to increase public awareness about transport issues and is intended to be the first of a series of publications on transport and energy issues in the Scottish context. The references form an important part of the report and provide a potentially important bibliography which must be augmented and updated regularly

Inverse simulation and analysis of underwater vehicle dynamics using feedback principles

Inverse simulation is a technique used in the modelling of dynamic systems that allows time histories of input variables to be found that generate required model output responses and provide inverse solutions in cases where analytical approaches to model inversion can present difficulties. This paper describes the application of inverse simulation to a nonlinear dynamic model of an underwater vehicle (UUV) and the determination of vehicle control inputs for specified manoeuvres. The approach to inverse simulation used in this application is based on the principles of feedback. Design issues relating to the UUV control surfaces and propeller thrust are highlighted through this procedure. The paper includes an outline of the nonlinear model of the UUV and typical sets of experimental conditions. Feedback loops are designed around the model for selected output variables and the inverse solutions are generated through simulation of this multi-input multi-output closed-loop system. It is shown that the feedback approach can provide inverse solutions for an appropriate choice of loop gain factors and integration time step using a fixed-step integration algorithm. Inverse solutions generated in this way are shown provide insight concerning issues of vehicle handling and manoeuvrability in a more direct fashion than is possible using conventional simulation methods

Methods of system identification, parameter estimation and optimisation applied to problems of modelling and control in engineering and physiology

Mathematical and computer-based models provide the foundation of most methods of engineering design. They are recognised as being especially important in the development of integrated dynamic systems, such as “control-configured” aircraft or in complex robotics applications. These models usually involve combinations of linear or nonlinear ordinary differential equations or difference equations, partial differential equations and algebraic equations. In some cases models may be based on differential algebraic equations. Dynamic models are also important in many other fields of research, including physiology where the highly integrated nature of biological control systems is starting to be more fully understood. Although many models may be developed using physical, chemical, or biological principles in the initial stages, the use of experimentation is important for checking the significance of underlying assumptions or simplifications and also for estimating appropriate sets of parameters. This experimental approach to modelling is also of central importance in establishing the suitability, or otherwise, of a given model for an intended application – the so-called “model validation” problem. System identification, which is the broad term used to describe the processes of experimental modelling, is generally considered to be a mature field and classical methods of identification involve linear discrete-time models within a stochastic framework. The aspects of the research described in this thesis that relate to applications of identification, parameter estimation and optimisation techniques for model development and model validation mainly involve nonlinear continuous time models Experimentally-based models of this kind have been used very successfully in the course of the research described in this thesis very in two areas of physiological research and in a number of different engineering applications. In terms of optimisation problems, the design, experimental tuning and performance evaluation of nonlinear control systems has much in common with the use of optimisation techniques within the model development process and it is therefore helpful to consider these two areas together. The work described in the thesis is strongly applications oriented. Many similarities have been found in applying modelling and control techniques to problems arising in fields that appear very different. For example, the areas of neurophysiology, respiratory gas exchange processes, electro-optic sensor systems, helicopter flight-control, hydro-electric power generation and surface ship or underwater vehicles appear to have little in common. However, closer examination shows that they have many similarities in terms of the types of problem that are presented, both in modelling and in system design. In addition to nonlinear behaviour; most models of these systems involve significant uncertainties or require important simplifications if the model is to be used in a real-time application such as automatic control. One recurring theme, that is important both in the modelling work described and for control applications, is the additional insight that can be gained through the dual use of time-domain and frequency-domain information. One example of this is the importance of coherence information in establishing the existence of linear or nonlinear relationships between variables and this has proved to be valuable in the experimental investigation of neuromuscular systems and in the identification of helicopter models from flight test data. Frequency-domain techniques have also proved useful for the reduction of high-order multi-input multi-output models. Another important theme that has appeared both within the modelling applications and in research on nonlinear control system design methods, relates to the problems of optimisation in cases where the associated response surface has many local optima. Finding the global optimum in practical applications presents major difficulties and much emphasis has been placed on evolutionary methods of optimisation (both genetic algorithms and genetic programming) in providing usable methods for optimisation in design and in complex nonlinear modelling applications that do not involve real-time problems. Another topic, considered both in the context of system modelling and control, is parameter sensitivity analysis and it has been found that insight gained from sensitivity information can be of value not only in the development of system models (e.g. through investigation of model robustness and the design of appropriate test inputs), but also in feedback system design and in controller tuning. A technique has been developed based on sensitivity analysis for the semi-automatic tuning of cascade and feedback controllers for multi-input multi-output feedback control systems. This tuning technique has been applied successfully to several problems. Inverse systems also receive significant attention in the thesis. These systems have provided a basis for theoretical research in the control systems field over the past two decades and some significant applications have been reported, despite the inherent difficulties in the mathematical methods needed for the nonlinear case. Inverse simulation methods, developed initially by others for use in handling-qualities studies for fixed-wing aircraft and helicopters, are shown in the thesis to provide some important potential benefits in control applications compared with classical methods of inversion. New developments in terms of methodology are presented in terms of a novel sensitivity based approach to inverse simulation that has advantages in terms of numerical accuracy and a new search-based optimisation technique based on the Nelder-Mead algorithm that can handle inverse simulation problems involving hard nonlinearities. Engineering applications of inverse simulation are presented, some of which involve helicopter flight control applications while others are concerned with feed-forward controllers for ship steering systems. The methods of search-based optimisation show some important advantages over conventional gradient-based methods, especially in cases where saturation and other nonlinearities are significant. The final discussion section takes the form of a critical evaluation of results obtained using the chosen methods of system identification, parameter estimation and optimisation for the modelling and control applications considered. Areas of success are highlighted and situations are identified where currently available techniques have important limitations. The benefits of an inter-disciplinary and applications-oriented approach to problems of modelling and control are also discussed and the value in terms of cross-fertilisation of ideas resulting from involvement in a wide range of applications is emphasised. Areas for further research are discussed

Some Issues in the Testing of Computer Simulation Models

The testing of simulation models has much in common with testing processes in other types of application involving software development. However, there are also important differences associated with the fact that simulation model testing involves two distinct aspects, which are known as verification and validation. Model validation is concerned with investigation of modelling errors and model limitations while verification involves checking that the simulation program is an accurate representation of the mathematical and logical structure of the underlying model. Success in model validation depends upon the availability of detailed information about all aspects of the system being modelled. It also may depend on the availability of high quality data from the system which can be used to compare its behaviour with that of the corresponding simulation model. Transparency, high standards of documentation and good management of simulation models and data sets are basic requirements in simulation model testing. Unlike most other areas of software testing, model validation often has subjective elements, with potentially important contributions from face- validation procedures in which experts give a subjective assessment of the fidelity of the model. Verification and validation processes are not simply applied once but must be used repeatedly throughout the model development process, with regressive testing principles being applied. Decisions about when a model is acceptable for the intended application inevitably involve some form of risk assessment. A case study concerned with the development and application of a simulation model of a hydro-turbine and electrical generator system is used to illustrate some of the issues arising in a typical control engineering application. Results from the case study suggest that it is important to bring together objective aspects of simulation model testing and the more subjective face- validation aspects in a coherent fashion. Suggestions are also made about the need for changes in approach in the teaching of simulation techniques to engineering students to give more emphasis to issues of model quality, testing and validation

Issues of fitness for purpose in train simulation models: a review

Many simulation models representing the longitudinal dynamics of a train are based on a single point-mass description. This leads to a second-order nonlinear ordinary differential equation, together with algebraic relationships. More complex multi-mass models may be used for models representing long trains involving many separate vehicles. However, in both cases, accuracy is limited by important underlying assumptions, approximations and parametric uncertainties. Another important aspect of train models concerns the direction of information flow. Input variables within conventional train models may represent power or tractive force, with acceleration, speed and distance travelled as output variables. However, inverse simulation methods can also be used, with the required speed or distance as inputs and tractive force, power, or energy as outputs. This allows energy requirements to be established for a given schedule and is useful when investigating fuel or energy economy. Inverse methods can also be used in powertrain design, such as for hybrid hydrogen fuel-cell/battery-electric trains. Issues of fitness for purpose are important in all such applications, both in terms of model uncertainties and in the additional insight offered by inverse simulation methods

Possible remnants of a frozen mud lake in southern Elysium, Mars

In this work we estimate the minimum persistence time of subsurface ice in water rich sediment layers remaining after sublimation of a martian lake. We simulate sublimation of ice from layers of different granulations and thicknesses. Presented results assume insolation and atmospheric conditions characteristic for the present day southern Elysium, where data from Mars Express have identified surface features possibly indicating the very recent presence of a frozen body of water [Murray et al., 2005. Nature 434, 352–356]. The age of these features is estimated to be several million years. On this time scale, we find that most of the water ice must have sublimated away, however remnant ice at a few percent level cannot be excluded. This amount of water ice is sufficient for chemical cementation of the observed features and explains their relatively pristine appearance, without significant signs of erosion

The application of parameter sensitivity analysis methods to inverse simulation models

Knowledge of the sensitivity of inverse solutions to variation of parameters of a model can be very useful in making engineering design decisions. This paper describes how parameter sensitivity analysis can be carried out for inverse simulations generated through approximate transfer function inversion methods and also by the use of feedback principles. Emphasis is placed on the use of sensitivity models and the paper includes examples and a case study involving a model of an underwater vehicle. It is shown that the use of sensitivity models can provide physical understanding of inverse simulation solutions that is not directly available using parameter sensitivity analysis methods that involve parameter perturbations and response differencing

Hybrid trains for the Highlands? Computer Simulations of Fuel-cell/Battery-electric Trains on Secondary Routes in Scotland

Although electrification is the preferred choice in the decarbonisation of railways in the United Kingdom there are important secondary routes where the business case for electrification is not strong. Examples include the lines north and west of Inverness and the West Highland lines linking central Scotland to Oban, Fort William and Mallaig. These routes all involve relatively long journeys, with few intermediate stations, prolonged gradients and many speed restrictions. Hydrogen fuel-cell/battery-electric hybrid units offer a possible solution for de-carbonisation of passenger services on lines such as these and, early in 2020, Transport Scotland and Scottish Enterprise announced financial support for development of a hydrogen fuel-cell/battery-electric multiple unit for trials in Scotland. The Hydrogen Accelerator group at St Andrews University is involved in management of the project and a contract for converting a former ScotRail Class 314 three-coach electric multiple unit to a hybrid configuration has been awarded to a group of companies led by Arcola Energy Ltd. This project forms part of a more broadly-based move to strengthen relevant industrial and business supply chains within the rail transport sector in Scotland and help promote new industrial/academic collaborations. Hydrogen fuel-cell stacks are characterised by a sluggish response to demanded power-level changes and their efficiency depends on the operating condition. Powertrain control strategies may therefore involve fuel-cell stack operation with slow rates of change that capture power demand, with fast dynamic changes and peak loads being supplied by the battery pack. The battery pack recharges through regenerative braking or from available power from the fuel-cell stack. Optimal powertrain component sizes depend on route characteristics, with relatively flat routes and operation at constant speed favouring large fuel-cells, while routes with prolonged and steep gradients or larger accelerations require larger batteries. Specifications for lengthy routes involving steep and prolonged gradients such as those encountered in the Scottish Highlands present significant difficulties. Mathematical models for longitudinal train motion involving second-order nonlinear ordinary differential equations, derived using Newton’s second law, provide a basis for conventional forward simulation of a train. Power or tractive force variables are applied as an input, with acceleration, speed and distance travelled being defined as outputs. In contrast, the analysis of road-vehicle powertrains often involves a reverse procedure which starts from a duty cycle based on a record of speed versus time with static or quasi-static models being used to estimate steady-state power and energy demands, However, although also involving an inverse type of approach, the simulation methods applied in this paper are based on dynamic models which allow transient power requirements to be understood more fully. These methods have been applied to the assessment of hydrogen train designs for some specific routes and also for test routes having profiles that are chosen to be typical of the routes of interest, but with simplified profiles. It is believed that use of these simplified test routes provides useful physical insight regarding the effect of train characteristics on the specifications for fuel cell stacks, battery packs and other powertrain components. Use of inverse modelling techniques has been found to allow straightforward investigation of performance sensitivities, not only in terms of the longitudinal train dynamics but also the powertrain parameters and route characteristics. Trade-off investigations using these inverse simulation models can be used to reduce the weight and volume of powertrain components and the cost of the train. Fuel-cell efficiency can also be considered, as larger cells allow operation over a wider range of conditions. Findings from the test routes considered allow estimation of powertrain ratings and storage requirements for operation of a three-coach hybrid hydrogen fuel-cell/battery electric train on the Glasgow to Fort William line

Simulation studies relating to rudder roll stabilization of a container ship using neural networks

RRS (Rudder Roll Stabilization) of Ships is a difficult problem because of its associated non-linear dynamics, coupling effects and complex control requirements. This paper proposes a solution of this stabilization problem that is based on an ANN (Artificial Neural Network) controller. The controller has been trained using supervised learning. The simulation studies have been carried out using MATLAB and a non-linear model of a container ship. It has been demonstrated that the proposed controller regulates heading and also controls roll angle very successfully